1. Field of the Invention
The present invention relates to an image processing apparatus, and more particularly, to an image processing apparatus which extracts, as a representative image, a frame from a moving image and sets it as an index.
2. Related Background Art
Conventionally, as a method of forming an index image to perform browsing and retrieval of moving images, there is the method in which one frame in the moving image is extracted and set as a representative image (called a “key frame” hereinafter), and this representative image is used as the index image together with the position thereof in the moving image.
The moving image can be divided into plural units each called a “shot”. The shot corresponds to a period from a photographing start to a photographing end in case of that the moving image is picked up by a video camera, and corresponds to a period (one scene) until a camera angle or the like is switched over in a case where the moving image is of television broadcasting. Thus, the method of detecting the switchover of the shot and setting the headmost frame of each shot as the index image is well known.
However, in fact, if two or more events are picked up because the camera is panned during one shot, in order to grasp the contents of this shot, it is insufficient to set only the headmost frame as the key frame. Moreover, for example, in a case where a zoom-in operation starts from the head of the shot and then continues until the end of the shot to zoom in on an object of interest, the frame after the zoom-in operation might be more suitable than the headmost frame as the key frame for the entire shot.
As a conventional technique for solving such problem, there is a method of extracting the frame after the zoom-in operation as the key frame by using camera operation information.
As an example of this method, Japanese Patent Application Laid-Open No. 7-99605 (corresponding to U.S. Pat. No. 5,581,362) discloses the technique that, when an image is picked up, the camera operation information is multiplexed or combined with video information and stored in a storage medium, and then, when the stored video information is reproduced, the frame satisfying a specific condition, e.g., after the zoom-in operation, is set as the key frame. Moreover, Japanese Patent Application Laid-Open No. 10-224736 discloses the technique that camera operation information is discriminated in an image process, and also the frame satisfying a specific condition, e.g., after the zoom-in operation, is set as the key frame.
Particularly, when an ordinary user picks up an image by using a video camera or the like, a smooth operation is frequently difficult for this user because the user is inexperienced in the camera's operation. For example, in a case of performing the zoom-in operation, the user has to repeat the zoom-in operations many times until the target image is enlarged to the desired size, and moreover has to perform a zoom-out operation also. This is because, since it is difficult to stop the zoom-in operation just in the desired size, the user has to correct the excessively zoomed-out image.
Here, such zoom-in and zoom-out operations should be originally considered as a series of operations. However, in Japanese Patent Application Laid-Open No. 7-99605, the end of the operation is detected in a case where, after operation information continuously increases or decreases for a certain number of frames or more, this information is continuously maintained without change for a certain number of frames.
For this reason, since such an operation as repeatedly increasing and decreasing the operation amount, e.g., repeatedly zooming in and out the images, cannot be considered as a series of operations, there is a problem that a number of similar images and meaningless images are extracted as key frames.
Moreover, if a user erroneously handles a zoom button and thus returns an operation state to its former state, where this operation is not intended by the user, then if the key frame is extracted, this frame merely represents a meaningless image.
However, in Japanese Patent Applications Laid-Open Nos. 7-99605 and 10-224736, there is a problem that the key frame is extracted in such instances of operations not intended by the user, or erroneous operation as above is performed.